Ultrasound-Based Force Sensing and Touch Sensing

ABSTRACT

An input/output device for a computing device including one or more touch sensors and one or more force sensors. The touch sensors sense data including one or more locations at which a contact or near-contact occurs. The force sensor sense data including a measure of an amount of force presented at the one or more locations at which a contact occurs. The touch sensors and the force sensors responsive to signals occurring in response to whether the signals are in response to contact or in response to an amount of force. The input/output device also includes one or more circuits coupled to the touch sensors and to the force sensors, and capable of combining information from both sensors.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/730,370, filed Jun. 4, 2015, and entitled “Ultrasound-Based ForceSensing and Touch Sensing,” which is a continuation of U.S. patentapplication Ser. No. 14/417,324, filed on Jan. 26, 2015, and entitled“Ultrasound-Based Force Sensing and Touch Sensing,” now abandoned, whichapplication is a 35 U.S.C. §371 application of PCT/US2013/032478, whichwas filed on Mar. 15, 2013, and entitled “Ultrasound-Based Force Sensingand Touch Sensing,” and further claims the benefit under 35 U.S.C.§119(e) to U.S. provisional application No. 61/676,297, filed Jul. 26,2012, and entitled, “Ultrasound-Based Force Sensing and Touch Sensing,”all of which are incorporated by reference as if fully disclosed herein.

BACKGROUND Field of the Disclosure

This application generally relates to force sensing on touch devicesusing ultrasound.

Background of the Disclosure

Touch devices generally provide for identification of positions wherethe user touches the device, including movement, gestures, and othereffects of position detection. For a first example, touch devices canprovide information to a computing system regarding user interactionwith a graphical user interface (GUI), such as pointing to elements,reorienting or repositioning those elements, editing or typing, andother GUI features. For a second example, touch devices can provideinformation to a computing system suitable for a user to interact withan application program, such as relating to input or manipulation ofanimation, photographs, pictures, slide presentations, sound, text,other audiovisual elements, and otherwise.

It sometimes occurs that, when interfacing with a GUI, or with anapplication program, it would be advantageous for the user to be able toindicate an amount of force applied when manipulating, moving, pointingto, touching, or otherwise interacting with, a touch device. Forexample, it might be advantageous for the user to be able to manipulatea screen element or other object in a first way with a relativelylighter touch, or in a second way with a relatively more forceful orsharper touch. In one such case, a it might be advantageous if the usercould move a screen element or other object with a relatively lightertouch, while the user could alternatively invoke or select that samescreen element or other object with a relatively more forceful orsharper touch.

It also sometimes occurs that, when interfacing with a GUI, or with anapplication program, it would be advantageous for the GUI or applicationprogram to be able to determine an identity of the user, or an affectthereof, in response to a degree of force applied by the user when usingthe touch device. For example, it might occur that a first user and asecond user differ in detectable ways with respect to how much forcethey apply, or when they apply that force, when using that GUI orapplication program.

Each of these examples, as well as other possible considerations, cancause one or more difficulties for the touch device, at least in thatinability to determine an amount of force applied by the user whencontacting the touch device might cause a GUI or an application programto be unable to provide functions that would be advantageous. When suchfunctions are called for, inability to provide those functions maysubject the touch device to lesser capabilities, to the possibledocument of the effectiveness and value of the touch device.

BRIEF SUMMARY OF THE DISCLOSURE

This application provides techniques, including circuits and designs,which can determine an amount of force applied, and changes in amountsof force applied, by the user when contacting a touch pad, and which canbe incorporated into devices using touch recognition, touch elements ofa GUI, and touch input or manipulation in an application program. Thisapplication also provides techniques, including devices which applythose techniques, which can determine an amount of force applied, andchanges in amounts of force applied, by the user when contacting a touchpad, and in response thereto, provide additional functions available toa user of a touch pad.

In one embodiment, techniques can include providing a force sensitivesensor incorporated into a touch device. For example, a force sensitivesensor can include an ultrasound device which can detect a measure ofhow forcefully a user is pressing, pushing, or otherwise contacting atouch device.

In one embodiment, the force sensitive sensor operates independently ofa second modality that determines one or more locations where the useris contacting the touch device, such as a capacitive touch sensor orother touch sensor. For example, a capacitive touch sensor can determineapproximately in what location the user is contacting the touch device,while an ultrasound device can detect how forcefully the user iscontacting the touch device.

In one embodiment, the force sensitive sensor includes one or more rowsand one or more columns, the rows and columns being disposed tointersect in a set of individual force sense elements. For example, theindividual force sense elements can be located in a substantiallyrectilinear array, with the rows disposed to define the individual rowsof that rectilinear array, the columns disposed to define the individualcolumns of that rectilinear array, and the intersections of the rows andcolumns disposed to define the individual elements of that rectilineararray.

In one embodiment, the rows and columns can be disposed so that each rowis controlled by a drive signal, each column is sensed by a sensecircuit, and the intersections between each row and each column aredisposed to generate and receive ultrasonic signals. For example theultrasonic signals can include, first, an ultrasound wave which isdirected at a possible position where the user might apply force to thetouch screen, and second, an ultrasound wave which is reflected fromthat position where the user actually does apply force to the touchscreen. In one embodiment, techniques can include providing a touchsensitive sensor, in addition to the force sensitive sensor, which candetermine a location where the user is actually touching the touchscreen. For example, the touch sensitive sensor can include a capacitivesensor, which can determine a location of the user's touch (such as bythe user's finger, another part of the user's body, or a stylus or otherobject).

In alternative embodiments, the force sensitive sensor can include a setof force sensors disposed in an arrangement other than a set of rows andcolumns disposed to intersect in a set of individual force senseelements. For a first example, the force sensitive sensor can include aset of individual sensor elements whose operation is not necessarily dueto intersection of rows and columns. For a second example, the forcesensitive sensor can include a set of individual sensor elementsdisposed in an array or other pattern, which might include a rectilinearpattern or another pattern.

In alternative embodiments, the force sensitive sensor can include a setof individual sensor elements which are disposed in a pattern thatallows force of touch to be detected, as to both location and amount, bymultiple individual sensor elements operating in concert. A set ofindividual sensor elements can be each disposed to determine force oftouch at a relative distance, and operate in conjunction so as todetermine location and amount of that force of touch.

In one embodiment, the touch sensitive sensor and the force sensitivesensor can include separate circuits, components, elements, modules, orotherwise, which can operate in combination or conjunction to separatelydetermine a location of touch and a force-of-touch. For example, asystem including the touch panel, an operating system program, anapplication program, a user interface, or otherwise, can be responsiveto the location of touch, the force-of-touch, a combination orconjunction of the two, or other factors.

One sample embodiment may take the form of a touch input/output (I/O)device, including: one or more touch sensors capable of providing touchlocation information, the touch location information including one ormore locations at which a contact or near-contact occurs; one or moreforce sensors capable of providing force of touch information, the forceof touch information including a measure of an amount of force presentedat the one or more locations at which a contact or near-contact occurs,wherein the one or more ultrasonic force sensors are separate from theone or more touch sensors; the touch sensors and the force sensorsresponsive to signals occurring at discernible times, the signalsindicative of contact; one or more circuits coupled to the touch sensorsand to the ultrasonic force sensors, and capable of combininginformation from the touch sensors and from the ultrasonic forcesensors.

Another sample embodiment may take the form of a method, includingoperations of providing touch location information in response to one ormore touch sensors, the touch location information including one or morelocations at which a contact or near-contact occurs; providing force oftouch information in response to one or more ultrasonic force sensors,the force of touch information including a measure of an amount of forcepresented at the one or more locations at which a contact ornear-contact occurs; providing the signals at discernible times inresponse to whether the signals are in response to contact or inresponse to an amount of force; and combining information from the touchsensors and from the ultrasonic force sensors; wherein the one or moreultrasonic force sensors is discrete from the one or more touch sensors.

For further examples, systems as described above can include the touchsensitive sensor, the force sensitive sensor, as well as other sensors,such as a mouse, trackpad, fingerprint sensor, biometric sensor, voiceactivation or voice recognition sensor, facial recognition sensor, orotherwise.

While multiple embodiments are disclosed, including variations thereof,still other embodiments of the present disclosure will become apparentto those skilled in the art from the following detailed description,which shows and describes illustrative embodiments of the disclosure. Aswill be realized, the disclosure is capable of modifications in variousobvious aspects, all without departing from the spirit and scope of thepresent disclosure. Accordingly, the drawings and detailed descriptionare to be regarded as illustrative in nature and not restrictive.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A is a front perspective view of a first example of a computingdevice incorporating a force sensing device.

FIG. 1B is a front perspective view of a second example of a computingdevice incorporating a force sensing device.

FIG. 1C is a front elevation view of a third example of a computingdevice incorporating the force sensing device.

FIG. 2 is a simplified cross-section view of the computing device takenalong line 2-2 in FIG. 1A.

FIG. 3 shows a conceptual drawing of communication between a touch I/Odevice and a computing system.

FIG. 4 shows a conceptual drawing of a system including a touch sensingand force sensing I/O device.

FIG. 5A shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including a liquidcrystal display (LCD) construction option.

FIG. 5B shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including aplastic organic light-emitting diode (OLED) construction option.

FIG. 6A shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including rowdrivers and sense columns.

FIG. 6B shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including signalsassociated with row drivers and sense columns.

FIG. 7A is a first example of a timing diagram for the computing device.

FIG. 7B is a second example of a timing diagram for the computingdevice.

FIG. 7C is a third example of a timing diagram for the computing device.

DETAILED DESCRIPTION

Terminology

The following terminology is exemplary, and not intended to be limitingin any way.

The text “touch sensing element”, and variants thereof, generally refersto one or more data sensing elements of any kind, including informationsensed with respect to individual locations. For example and withoutlimitation, a touch sensing element can sense data or other informationwith respect to a relatively small region of where a user is contactinga touch device.

The text “force sensing element”, and variants thereof, generally refersto one or more data sensing elements of any kind, including informationsensed with respect to force-of-touch, whether at individual locationsor otherwise. For example and without limitation, a force sensingelement can sense data or other information with respect to a relativelysmall region of where a user is forcibly contacting a device.

The text “force-of-touch”, and variants thereof, generally refers to adegree or measure of an amount of force being applied to a device. Thedegree or measure of an amount of force need not have any particularscale; for example, the measure of force-of-touch can be linear,logarithmic, or otherwise nonlinear, and can be adjusted periodically(or otherwise, such as a periodically or otherwise from time to time) inresponse to one or more factors, either relating to force-of-touch,location of touch, time, or otherwise.

After reading this application, it should be noted that these statementsof terminology would be applicable to techniques, methods, physicalelements, and systems (whether currently known or otherwise), includingextensions thereof inferred or inferable by those skilled in the artafter reading this application.

Overview

The present disclosure is generally related to a force sensing devicethat may be incorporated into a variety of electronic or computingdevices, such as, but not limited to, computers, smart phones, tabletcomputers, track pads, and so on. The force sensing device may be usedto detect one or more user force inputs on an input surface and then aprocessor (or processing element) may correlate the sensed inputs into aforce measurement and provide those inputs to the computing device. Insome embodiments, the force sensing device may be used to determineforce inputs to a track pad, a display screen, or other input surface.

The force sensing device may include an input surface, a force sensingmodule, a substrate or support layer, and optionally a sensing layerthat may detect another input characteristic than the force sensinglayer. The input surface provides an engagement surface for a user, suchas the external surface of a track pad or the cover glass for a display.In other words, the input surface may receive one or more user inputsdirectly or indirectly.

The force sensing module may include an ultrasonic module or sensor thatmay emit and detect ultrasonic pulses. In one example, the ultrasonicmodule may include a plurality of sensing elements arranged in rows orcolumns, where each of the sensing elements may selectively emit anultrasonic pulse or other signal. The pulse may be transmitted throughthe components of the force sensing device, such as through the sensinglayer and the input surface. When the pulse reaches the input surface,it may be reflected by a portion of the user (e.g., finger) or otherobject, which may reflect the pulse. The reflection of the pulse mayvary based on distance that the particular sensing element receiving thepulse is from the input. Additionally, the degree of attenuation of thepulse may also be associated with a force magnitude associated with theinput. For example, generally, as the input force on the input surfaceincreases, the contacting object exerting the force may absorb a largerpercentage of the pulse, such that the reflected pulse may be diminishedcorrespondingly.

In embodiments where it is present, the sensing layer may be configuredto sense characteristics different from the force sensing module. Forexample, the sensing layer may include capacitive sensors or othersensing elements. In a specific implantation, a multi-touch sensinglayer may be incorporated into the force sensing device and may be usedto enhance data regarding user inputs. As an example, touch inputsdetected by the sense layer may be used to further refine the forceinput location, confirm the force input location, and/or correlate theforce input to an input location. In the last example, the forcesensitive device may not use the capacitive sensing of the force sensingdevice to estimate a location, which may reduce the processing requiredfor the force sensing device. Additionally, in some embodiments, a touchsensitive device may be used to determine force inputs for a number ofdifferent touches. For example, the touch positions and force inputs maybe used to estimate the input force at each touch location.

Force Sensitive Device and System

Turning now to the figures, illustrative electronic devices that mayincorporate the force sensing device will be discussed in more detail.FIGS. 1A-1C illustrate various computing or electronic devices that mayincorporate the force sensing device. With reference to FIG. 1A, theforce sensing device may be incorporated into a computer 10, such as alaptop or desktop computer. The computer 10 may include a track pad 12or other input surface, a display 14, and an enclosure 16 or frame. Theenclosure 16 may extend around a portion of the track pad 12 and/ordisplay 14. In the embodiment illustrated in FIG. 1A, the force sensingdevice may be incorporated into the track pad 12, the display 14, orboth the track pad 12 and the display 14. In these embodiments, theforce sensing device may be configured to detect force inputs to thetrack pad 12 and/or the display 14.

In some embodiments, the force sensing device may be incorporated into atablet computer. FIG. 1B is a top perspective view of a tablet computerincluding the force sensing device. With reference to FIG. 1B, the tablecomputer 10 may include the display 14 where the force sensing device isconfigured to detect force inputs to the display 14. In addition to theforce sensing device, the display 14 may also include one or more touchsensors, such as a multi-touch capacitive grid, or the like. In theseembodiments, the display 14 may detect both force inputs, as well asposition or touch inputs.

In yet other embodiments, the force sensing device may be incorporatedinto a mobile computing device, such as a smart phone. FIG. 1C is aperspective view of a smart phone including the force sensing device.With reference to FIG. 1C, the smart phone 10 may include a display 14and a frame or enclosure 16 substantially surrounding a perimeter of thedisplay 14. In the embodiment illustrated in FIG. 1C, the force sensingdevice may be incorporated into the display 14. Similarly to theembodiment illustrated in FIG. 1B, in instances where the force sensingdevice may be incorporated into the display 14, the display 14 may alsoinclude one or more position or touch sensing devices in addition to theforce sensing device.

The force sensing device will now be discussed in more detail. FIG. 2 isa simplified cross-section view of the electronic device taken alongline 2-2 in FIG. 1A. With reference to FIG. 2, the force sensing device18 may include an input surface 20, a sensing layer 22, a force sensingmodule 24 or layer, and a substrate 28. As discussed above with respectto FIGS. 1A-1C, the input surface 20 may form an exterior surface (or asurface in communication with an exterior surface) of the track pad 12,the display 14, or other portions (such as the enclosure) of thecomputing device 10. In some embodiments, the input surface 20 may be atleast partially translucent. For example, in embodiments where the forcesensing device 18 is incorporated into a portion of the display 14.

The sensing layer 22 may be configured to sense one or more parameterscorrelated to a user input. In some embodiments, the sensing layer 22may be configured to sense characteristics or parameters that may bedifferent from the characteristics sensed by the force sensing module24. For example, the sensing layer 22 may include one or more capacitivesensors that may be configured to detect input touches, e.g.,multi-touch input surface including intersecting rows and columns. Thesensing layer 22 may be omitted where additional data regarding the userinputs may not be desired. Additionally, the sensing layer 22 mayprovide additional data that may be used to enhance data sensed by theforce sensing module 24 or may be different from the force sensingmodule. In some embodiments, there may be an air gap between the sensinglayer 22 and the force sensing module 24. In other words, the forcesensing module 24 and sensing layer may be spatially separated from eachother defining a gap or spacing distance.

The substrate 28 may be substantially any support surface, such as aportion of an printed circuit board, the enclosure 16 or frame, or thelike. Additionally, the substrate 28 may be configured to surround or atleast partially surround one more sides of the sensing device 18.

In some embodiments, a display (e.g., a liquid crystal display) may bepositioned beneath the input surface 20 or may form a portion of theinput surface 20. Alternatively, the display may be positioned betweenother layers of the force sensing device. In these embodiments, visualoutput provided by the display may be visible through the input surface20.

As generally discussed above, the force sensing device may beincorporated into one or more touch sensitive device. FIG. 3 shows aconceptual drawing of communication between a touch I/O device and acomputing system. FIG. 4 shows a conceptual drawing of a systemincluding a force sensitive touch device.

Described embodiments may include touch I/O device 1001 that can receivetouch input and force input (such as possibly including touch locationsand force of touch at those locations) for interacting with computingsystem 1003 (such as shown in the FIGS. 1A-1C) via wired or wirelesscommunication channel 1002. Touch I/O device 1001 may be used to provideuser input to computing system 1003 in lieu of or in combination withother input devices such as a keyboard, mouse, or possibly otherdevices. In alternative embodiments, touch I/O device 1001 may be usedin conjunction with other input devices, such as in addition to or inlieu of a mouse, trackpad, or possibly another pointing device. One ormore touch I/O devices 1001 may be used for providing user input tocomputing system 1003. Touch I/O device 1001 may be an integral part ofcomputing system 1003 (e.g., touch screen on a laptop) or may beseparate from computing system 1003.

Touch I/O device 1001 may include a touch sensitive and force sensitivepanel which is wholly or partially transparent, semitransparent,non-transparent, opaque or any combination thereof. Touch I/O device1001 may be embodied as a touch screen, touch pad, a touch screenfunctioning as a touch pad (e.g., a touch screen replacing the touchpadof a laptop), a touch screen or touchpad combined or incorporated withany other input device (e.g., a touch screen or touchpad disposed on akeyboard, disposed on a trackpad or other pointing device), anymulti-dimensional object having a touch sensitive surface for receivingtouch input, or another type of input device or input/output device.

In one example, touch I/O device 1001 embodied as a touch screen mayinclude a transparent and/or semitransparent touch sensitive and forcesensitive panel at least partially or wholly positioned over at least aportion of a display. (Although the touch sensitive and force sensitivepanel is described as at least partially or wholly positioned over atleast a portion of a display, in alternative embodiments, at least aportion of circuitry or other elements used in embodiments of the touchsensitive and force sensitive panel may be at least positioned partiallyor wholly positioned under at least a portion of a display, interleavedwith circuits used with at least a portion of a display, or otherwise.)According to this embodiment, touch I/O device 1001 functions to displaygraphical data transmitted from computing system 1003 (and/or anothersource) and also functions to receive user input. In other embodiments,touch I/O device 1001 may be embodied as an integrated touch screenwhere touch sensitive and force sensitive components/devices areintegral with display components/devices. In still other embodiments atouch screen may be used as a supplemental or additional display screenfor displaying supplemental or the same graphical data as a primarydisplay and to receive touch input, including possibly touch locationsand force of touch at those locations.

Touch I/O device 1001 may be configured to detect the location of one ormore touches or near touches on device 1001, and where applicable, forceof those touches, based on capacitive, resistive, optical, acoustic,inductive, mechanical, chemical, or electromagnetic measurements, inlieu of or in combination or conjunction with any phenomena that can bemeasured with respect to the occurrences of the one or more touches ornear touches, and where applicable, force of those touches, in proximityto deice 1001. Software, hardware, firmware or any combination thereofmay be used to process the measurements of the detected touches, andwhere applicable, force of those touches, to identify and track one ormore gestures. A gesture may correspond to stationary or non-stationary,single or multiple, touches or near touches, and where applicable, forceof those touches, on touch I/O device 1001. A gesture may be performedby moving one or more fingers or other objects in a particular manner ontouch I/O device 1001 such as tapping, pressing, rocking, scrubbing,twisting, changing orientation, pressing with varying pressure and thelike at essentially the same time, contiguously, consecutively, orotherwise. A gesture may be characterized by, but is not limited to apinching, sliding, swiping, rotating, flexing, dragging, tapping,pushing and/or releasing, or other motion between or with any otherfinger or fingers, or any other portion of the body or other object. Asingle gesture may be performed with one or more hands, or any otherportion of the body or other object by one or more users, or anycombination thereof.

Computing system 1003 may drive a display with graphical data to displaya graphical user interface (GUI). The GUI may be configured to receivetouch input, and where applicable, force of that touch input, via touchI/O device 1001. Embodied as a touch screen, touch I/O device 1001 maydisplay the GUI. Alternatively, the GUI may be displayed on a displayseparate from touch I/O device 1001. The GUI may include graphicalelements displayed at particular locations within the interface.Graphical elements may include but are not limited to a variety ofdisplayed virtual input devices including virtual scroll wheels, avirtual keyboard, virtual knobs or dials, virtual buttons, virtuallevers, any virtual UI, and the like. A user may perform gestures at oneor more particular locations on touch I/O device 1001 which may beassociated with the graphical elements of the GUI. In other embodiments,the user may perform gestures at one or more locations that areindependent of the locations of graphical elements of the GUI. Gesturesperformed on touch I/O device 1001 may directly or indirectlymanipulate, control, modify, move, actuate, initiate or generally affectgraphical elements such as cursors, icons, media files, lists, text, allor portions of images, or the like within the GUI. For instance, in thecase of a touch screen, a user may directly interact with a graphicalelement by performing a gesture over the graphical element on the touchscreen. Alternatively, a touch pad generally provides indirectinteraction. Gestures may also affect non-displayed GUI elements (e.g.,causing user interfaces to appear) or may affect other actions withincomputing system 1003 (e.g., affect a state or mode of a GUI,application, or operating system). Gestures may or may not be performedon touch I/O device 1001 in conjunction with a displayed cursor. Forinstance, in the case in which gestures are performed on a touchpad, acursor (or pointer) may be displayed on a display screen or touch screenand the cursor may be controlled via touch input, and where applicable,force of that touch input, on the touchpad to interact with graphicalobjects on the display screen. In other embodiments in which gesturesare performed directly on a touch screen, a user may interact directlywith objects on the touch screen, with or without a cursor or pointerbeing displayed on the touch screen.

Feedback may be provided to the user via communication channel 1002 inresponse to or based on the touch or near touches, and where applicable,force of those touches, on touch I/O device 1001. Feedback may betransmitted optically, mechanically, electrically, olfactory,acoustically, haptically, or the like or any combination thereof and ina variable or non-variable manner.

Attention is now directed towards embodiments of a system architecturethat may be embodied within any portable or non-portable deviceincluding but not limited to a communication device (e.g. mobile phone,smart phone), a multi-media device (e.g., MP3 player, TV, radio), aportable or handheld computer (e.g., tablet, netbook, laptop), a desktopcomputer, an All-In-One desktop, a peripheral device, or any other(portable or non-portable) system or device adaptable to the inclusionof system architecture 2000, including combinations of two or more ofthese types of devices. FIG. 4 is a block diagram of one embodiment ofsystem 2000 that generally includes one or more computer-readablemediums 2001, processing system 2004, Input/Output (I/O) subsystem 2006,electromagnetic frequency (EMF) circuitry (such as possibly radiofrequency or other frequency circuitry) 2008 and audio circuitry 2010.These components may be coupled by one or more communication buses orsignal lines 2003. Each such bus or signal line may be denoted in theform 2003-X, where X can be a unique number. The bus or signal line maycarry data of the appropriate type between components; each bus orsignal line may differ from other buses/lines, but may perform generallysimilar operations.

It should be apparent that the architecture shown in FIGS. 1A-4 is onlyone example architecture of system 2000, and that system 2000 could havemore or fewer components than shown, or a different configuration ofcomponents. The various components shown in FIGS. 1A-4 can beimplemented in hardware, software, firmware or any combination thereof,including one or more signal processing and/or application specificintegrated circuits.

EMF circuitry 2008 is used to send and receive information over awireless link or network to one or more other devices and includeswell-known circuitry for performing this function. EMF circuitry 2008and audio circuitry 2010 are coupled to processing system 2004 viaperipherals interface 2016. Interface 2016 includes various knowncomponents for establishing and maintaining communication betweenperipherals and processing system 2004. Audio circuitry 2010 is coupledto audio speaker 2050 and microphone 2052 and includes known circuitryfor processing voice signals received from interface 2016 to enable auser to communicate in real-time with other users. In some embodiments,audio circuitry 2010 includes a headphone jack (not shown).

Peripherals interface 2016 couples the input and output peripherals ofthe system to processor 2018 and computer-readable medium 2001. One ormore processors 2018 communicate with one or more computer-readablemediums 2001 via controller 2020. Computer-readable medium 2001 can beany device or medium that can store code and/or data for use by one ormore processors 2018. Medium 2001 can include a memory hierarchy,including but not limited to cache, main memory and secondary memory.The memory hierarchy can be implemented using any combination of RAM(e.g., SRAM, DRAM, DDRAM), ROM, FLASH, magnetic and/or optical storagedevices, such as disk drives, magnetic tape, CDs (compact disks) andDVDs (digital video discs). Medium 2001 may also include a transmissionmedium for carrying information-bearing signals indicative of computerinstructions or data (with or without a carrier wave upon which thesignals are modulated). For example, the transmission medium may includea communications network, including but not limited to the Internet(also referred to as the World Wide Web), intranet(s), Local AreaNetworks (LANs), Wide Local Area Networks (WLANs), Storage Area Networks(SANs), Metropolitan Area Networks (MAN) and the like.

One or more processors 2018 run various software components stored inmedium 2001 to perform various functions for system 2000. In someembodiments, the software components include operating system 2022,communication module (or set of instructions) 2024, touch andforce-of-touch processing module (or set of instructions) 2026, graphicsmodule (or set of instructions) 2028, one or more applications (or setof instructions) 2030, and fingerprint sensing module (or set ofinstructions) 2038. Each of these modules and above noted applicationscorrespond to a set of instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwiserearranged in various embodiments. In some embodiments, medium 2001 maystore a subset of the modules and data structures identified above.Furthermore, medium 2001 may store additional modules and datastructures not described above.

Operating system 2022 includes various procedures, sets of instructions,software components and/or drivers for controlling and managing generalsystem tasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 2024 facilitates communication with other devicesover one or more external ports 2036 or via EMF circuitry 2008 andincludes various software components for handling data received from EMFcircuitry 2008 and/or external port 2036.

Graphics module 2028 includes various known software components forrendering, animating and displaying graphical objects on a displaysurface. In embodiments in which touch I/O element 2012 is a touchsensitive and force sensitive display (e.g., touch screen), graphicsmodule 2028 includes components for rendering, displaying, and animatingobjects on the touch sensitive and force sensitive display.

One or more applications 2030 can include any applications installed onsystem 2000, including without limitation, a browser, address book,contact list, email, instant messaging, word processing, keyboardemulation, widgets, JAVA-enabled applications, encryption, digitalrights management, voice recognition, voice replication, locationdetermination capability (such as that provided by the globalpositioning system, also sometimes referred to herein as “GPS”), a musicplayer, and otherwise.

Touch and force-of-touch processing module 2026 includes varioussoftware components for performing various tasks associated with touchI/O element 2012 including but not limited to receiving and processingtouch input and force-of-touch input received from I/O device 2012 viatouch I/O element controller 2032.

System 2000 may further include fingerprint sensing module 2038 forperforming the method/functions as described herein in connection withother figures shown and described herein.

I/O subsystem 2006 is coupled to touch I/O element 2012 and one or moreother I/O devices 2014 for controlling or performing various functions.Touch I/O element 2012 communicates with processing system 2004 viatouch I/O element controller 2032, which includes various components forprocessing user touch input and force-of-touch input (e.g., scanninghardware). One or more other input controllers 2034 receives/sendselectrical signals from/to other I/O devices 2014. Other I/O devices2014 may include physical buttons, dials, slider switches, sticks,keyboards, touch pads, additional display screens, or any combinationthereof.

If embodied as a touch screen, touch I/O element 2012 displays visualoutput to the user in a GUI. The visual output may include text,graphics, video, and any combination thereof. Some or all of the visualoutput may correspond to user-interface objects. Touch I/O element 2012forms a touch-sensitive and force-sensitive surface that accepts touchinput and force-of- touch input from the user. Touch I/O element 2012and touch screen controller 2032 (along with any associated modulesand/or sets of instructions in medium 2001) detects and tracks touchesor near touches, and where applicable, force of those touches (and anymovement or release of the touch, and any change in the force of thetouch) on touch I/O element 2012 and converts the detected touch inputand force-of-touch input into interaction with graphical objects, suchas one or more user-interface objects. In the case in which device 2012is embodied as a touch screen, the user can directly interact withgraphical objects that are displayed on the touch screen. Alternatively,in the case in which device 2012 is embodied as a touch device otherthan a touch screen (e.g., a touch pad or trackpad), the user mayindirectly interact with graphical objects that are displayed on aseparate display screen embodied as I/O device 2014.

Touch I/O element 2012 may be analogous to the multi-touch sensitivesurface described in the following U.S. Pat. Nos. 6,323,846; 6,570,557;and/or 6,677,932; and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference.

Embodiments in which touch I/O element 2012 is a touch screen, the touchscreen may use LCD (liquid crystal display) technology, LPD (lightemitting polymer display) technology, OLED (organic LED), or OEL(organic electro luminescence), although other display technologies maybe used in other embodiments.

Feedback may be provided by touch I/O element 2012 based on the user'stouch, and force-of-touch, input as well as a state or states of what isbeing displayed and/or of the computing system. Feedback may betransmitted optically (e.g., light signal or displayed image),mechanically (e.g., haptic feedback, touch feedback, force feedback, orthe like), electrically (e.g., electrical stimulation), olfactory,acoustically (e.g., beep or the like), or the like or any combinationthereof and in a variable or non-variable manner.

System 2000 also includes power system 2044 for powering the varioushardware components and may include a power management system, one ormore power sources, a recharging system, a power failure detectioncircuit, a power converter or inverter, a power status indicator and anyother components typically associated with the generation, managementand distribution of power in portable devices.

In some embodiments, peripherals interface 2016, one or more processors2018, and memory controller 2020 may be implemented on a single chip,such as processing system 2004. In some other embodiments, they may beimplemented on separate chips.

Ultrasound-Based Sensing With Separate Touch Modules

Although this application primarily describes particular embodimentswith respect to configuration of the system including ultrasound-basedsensing, in the context of the invention, there is no particularrequirement for any limitation to those particular embodiments. Whileparticular elements are described for layering of elements in oneembodiment, alternative elements would also be workable.

For example, while this application primarily describes embodiments inwhich a set of ultrasound-based force sensing elements are disposedbelow a set of presentation elements and below a set of touch sensingelements, in alternative embodiments, there is no particular requirementfor that ordering of elements. For example, the ultrasound-based forcesensing elements could be disposed above the presentation elements andcould be constructed or arranged so they do not interfere with thepresentation elements, such as being translucent or transparent, or withthe presentation elements disposed between individual force sensingelements.

For example, the ultrasound-based force sensing elements could bedisposed above the presentation elements, but so arranged that the forcesensing elements are interspersed with the presentation elements, withthe effect that the presentation elements can present light and color toa user through the cover glass, without obstruction by any of the forcesensing elements.

LCD CONSTRUCTION OPTION. FIG. 5A shows a conceptual drawing of a systemincluding ultrasound-based sensing with separate touch modules,including a liquid crystal display (LCD) construction option 102.

A system including ultrasound-based sensing with separate touch modulesincludes a touch I/O element 2012 as described herein, including a coverglass (CG) element 104, which may be touched by the user, and for whichtouch may be sensed and force-of-touch may be sensed. Anultrasound-based force sensing element is disposed below the coverglass. A touch sensing element is also disposed below the cover glass.

In one embodiment, the touch I/O element 2012 can include a liquidcrystal display (LCD) construction option.

In one embodiment, the liquid crystal display (LCD) construction option102 can include the cover glass (CG) element 104, which can have athickness of approximately 600 microns. The cover glass (CG) element 104might be used to receive touch and force of touch from the user. Thecover glass (CG) element 104 can be constructed using one or more layersof glass, chemically treated glass, sapphire, or one or more othersubstances.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a first optically clear adhesive (OCA) element 106disposed below the cover glass element, which can have a thickness ofapproximately 150 microns. In alternative embodiments, other adhesiveelements which do not interfere with operation of the other elements ofthe system could be used.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a top point of load (POL) element 108 disposed below thefirst optically clear adhesive (OCA) 106 element, which can have athickness of approximately 200 microns. The top POL element might beused to distribute power to elements of the touch I/O element 2012.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a CF glass element 110 disposed below the top POLelement 108, which, in some implementations can have a thickness ofapproximately 150 microns.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a thin film transistor (TFT) LCD glass element 112disposed below the top POL element 108, which can have a thickness ofapproximately 150 microns. The TFT LCD element 112 might be used topresent display elements for the touch I/O element 2012.

Although this application primarily describes an embodiment using a TFTelement 112, which can have the capability of presenting an image to auser through the cover glass, in the context of the invention, manyalternatives exist which would also be workable. In alternativeembodiments, the TFT element 112 (in combination with the top POLelement 108 and the bottom POL element 114) can be disposed in anotherlocation in a stack of elements disposed below the cover glass 104. Forexample, the TFT can be disposed below the touch sensor, which can insuch cases be constructed of a transparent or translucent material, orotherwise disposed so that presentation of an image to a user can beperformed.

Moreover, although this application primarily describes an embodimentusing a TFT element 112 which is coupled to a top POL element 108 and abottom POL element 114, in the context of the invention, manyalternatives exist which would also be workable. In alternativeembodiments, the TFT element 112 could be used with a single layer forsignal distribution, such as a single layer which includes threeelectrodes for each TFT element, versus a top POL layer and a bottom POLlayer, each of which includes two electrodes for each TFT element.

In one embodiment, the liquid crystal display (LCD) construction optioncan include a bottom POL element 114 disposed below the TFT LCD glasselement 112, which can have a thickness of approximately 200 microns.Similar to the top POL element 108, the bottom POL 114 element might beused to distribute power to elements of the touch device 2012.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a second first optically clear adhesive (OCA) element116 disposed below the bottom POL element 114, which can have athickness of approximately 150 microns. As described above with respectto the first OCA element 106, in alternative embodiments, other adhesiveelements which do not interfere with operation of the other elements ofthe system could be used.

In one embodiment, the liquid crystal display (LCD) construction option102 can include an ultrasonic element 118, such as polyvinylidenedifluoride (PVDF), disposed below the second first optically clearadhesive (OCA) element 116, which can have a thickness of approximately50 microns.

Although this application primarily describes a system using a PVDF asthe ultrasonic element 118, in alternative embodiments, anypiezoelectric substance would also be workable, and is within the scopeand spirit of the invention. Moreover, any other substance which couldbe excited to generate a sonic pulse, such as an ultrasonic pulse whichcan result in reflection from the an interface between the surface ofthe cover glass and either the air or the user's finger, would also beworkable, and is within the scope and spirit of the invention. Moreover,any other substance which could be excited to generate a signal whichcan detect force of touch, such as a signal which could be absorbed, oralternatively, reflected, in response to a force of touch from a user'sfinger.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a set of backlight layers 120 disposed below theultrasonic element 118. The backlight layers, in combination with theTFT LCD glass element 112, can provide the touch I/O element 2012 with adisplay capability.

In one embodiment, the liquid crystal display (LCD) construction option102 can include a set of semi-transparent sense column circuits 122,disposed below the second first optically clear adhesive (OCA) element106. For example, the semi-transparent sense column circuits 122 caninclude one or more metal layers or metallized layers, in which circuitelements are disposed. In one such example, the circuit elements aredisposed in a set of horizontal and vertical wire elements, located soas not to cover the TFT LCD elements 112 (or other related displayelements), with the effect that the circuits do not cause the lightedelements of the display to be substantially hindered or obscured.Similarly, in one embodiment, the liquid crystal display (LCD)construction option 102 can include a second of semi-transparent rowdriver circuits 124, disposed below the ultrasonic element 118. Forexample, the semi-transparent row driver circuits 124 can also includeone or more metal layers or metallized layers, in which circuit elementsare disposed. In one such example, the circuit elements are disposed ina set of horizontal and vertical wire elements, located so as not tocover the TFT LCD elements 112 (or other related display elements), withthe effect that the circuits do not cause the lighted elements of thedisplay to be substantially hindered or obscured. The sense columncircuits 122 and the row driver circuits 124 are further describedbelow.

As described above, while this application describes a particularordering of layers, in alternative embodiments, other orderings would beworkable, and are within the scope and spirit of the invention.Similarly, as described above, other substances other than OCA would beworkable, and are within the scope and spirit of the invention.Similarly, as described above, other materials other than PVDF, such asother piezoelectric substances or other circuits or elements which couldgenerate a signal capable of reflection from an interface between thesurface of the cover glass and either the air or the user's finger, orotherwise detecting force of touch, would be workable, and are withinthe scope and spirit of the invention. Similarly, as described above,elements which are described to have a top and a bottom set of circuitsfor activation, would in alternative embodiments also be workable withonly a single layer of circuits for activation, such as a single layerusing three electrodes for activating individual elements, rather thantwo layers each having only two electrodes coupled to each element.Moreover, although certain thicknesses and other dimensions have beenprovided, it should be noted that these are meant as illustrative only.Many other thicknesses are envisioned and may be varied based on one ormore design parameters, such as overall thickness of the device, cost,and so on.

PLASTIC OLED CONSTRUCTION OPTION. FIG. 5B shows a conceptual drawing ofa system including ultrasound-based sensing with separate touch modules,including a plastic organic light-emitting diode (OLED) constructionoption 202.

A system including ultrasound-based sensing with separate touch modulesincludes a touch I/O element 2012 as described herein, including a coverglass (CG) element 204, which may be touched by the user, and for whichtouch may be sensed and force-of-touch may be sensed. Anultrasound-based force sensing element is disposed below the coverglass. A touch sensing element is also disposed below the cover glass.

In one embodiment, the touch I/O element 2012 can include a plasticorganic light-emitting diode (OLED) construction option 202.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include the cover glass (CG) element 204,which can have a thickness of approximately 600 microns. The cover glass(CG) element 204 might be used to receive touch and force of touch fromthe user. The cover glass (CG) element 204 can be constructed using oneor more layers of glass, chemically treated glass, sapphire, or one ormore other substances.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include a first optically clear adhesive(OCA) element 206 disposed below the cover glass element 204, which canhave a thickness of approximately 150 microns. In alternativeembodiments, other adhesive elements which do not interfere withoperation of the other elements of the system could be used.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include a plastic film dual indium-titaniumoxide (DITO) element 208 disposed below the first optically clearadhesive (OCA) element 206, which had thickness of approximately 115microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include a second optically clear adhesive(OCA) element 210 disposed below the plastic dual indium-titanium oxide(DITO) element 208, which can have a thickness of approximately 150microns. As described above, in alternative embodiments, other adhesiveelements which do not interfere with operation of the other elements ofthe system could be used.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include an OLED display element 212 disposedbelow the second optically clear adhesive (OCA) element. The OLEDdisplay element may be a plastic OLED and may have a thickness ofapproximately 250 microns. Although this application primarily describesan embodiment using a plastic OLED display element, which can have thecapability of presenting an image to a user through the cover glass, inthe context of the invention, many alternatives exist which would alsobe workable. In alternative embodiments, the plastic OLED displayelement (in combination with the DITO element) can be disposed inanother location in a stack of elements disposed below the cover glass.For example, the plastic OLED display element can be disposed below thetouch sensor, which can in such cases be constructed of a transparent ortranslucent material, or otherwise disposed so that presentation of animage to a user can be performed.

Moreover, although this application primarily describes an embodimentusing a plastic OLED display element which is coupled to a DITO element,in the context of the invention, many alternatives exist which wouldalso be workable. In alternative embodiments, the plastic OLED displayelement could be used with a single layer for signal distribution, suchas a single layer which includes three electrodes for each plastic OLEDdisplay element, versus two layers for signal distribution, each ofwhich includes two electrodes for each plastic OLED display element.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option 202 can include an ultrasonic element 216. In someembodiments, the ultrasonic element may be polyvinylidene difluoride(PVDF). Additionally, the ultrasonic element may be disposed below theplastic OLED display element, which can have a thickness ofapproximately 50 microns.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a foam or rubber element 220 disposedbelow the ultrasonic element 216, and a mid-plate element 222 (such as asupport element) disposed below the foam or rubber element 218.

In one embodiment, the plastic organic light-emitting diode (OLED)construction option can include a set of semi-transparent sense columncircuits 214, disposed below the second optically clear adhesive (OCA)element 210. For example, the semi-transparent sense column circuits 214can include one or more metal layers or metallized layers, in whichcircuit elements are disposed. In one such example, the circuit elementsare disposed in a set of horizontal and vertical wire elements, locatedso as not to cover the OLED elements 212 (or other related displayelements), with the effect that the circuits do not cause the lightedelements of the display to be substantially hindered or obscured.Similarly, in one embodiment, the plastic organic light-emitting diode(OLED) construction option 202 can include a second of semi-transparentrow driver circuits 218, disposed below the ultrasonic element 216. Forexample, the semi- transparent row driver circuits 218 can include oneor more metal layers or metallized layers, in which circuit elements aredisposed. In one such example, the circuit elements are disposed in aset of horizontal and vertical wire elements, located so as not to coverthe OLED elements 212 (or other related display elements), with theeffect that the circuits do not cause the lighted elements of thedisplay to be substantially hindered or obscured. The sense columncircuits and the row driver circuits are further described below.

As described above, while this application describes a particularordering of layers, in alternative embodiments, other orderings would beworkable, and are within the scope and spirit of the invention.Similarly, as described above, other substances other than OCA would beworkable, and are within the scope and spirit of the invention.Similarly, as described above, other materials other than PVDF, such asother piezoelectric substances or other circuits or elements which couldgenerate a signal capable of reflection from an interface between thesurface of the cover glass and either the air or the user's finger, orotherwise detecting force of touch, would be workable, and are withinthe scope and spirit of the invention. Similarly, as described above,elements which are described to have a top and a bottom set of circuitsfor activation, would in alternative embodiments also be workable withonly a single layer of circuits for activation, such as a single layerusing three electrodes for activating individual elements, rather thantwo layers each having only two electrodes coupled to each element.

Row and Column Circuits for Ultrasound-Based Sensing

FIG. 6A shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including rowdrivers and sense columns.

FIG. 6B shows a conceptual drawing of a system includingultrasound-based sensing with separate touch modules, including signalsassociated with row drivers and sense columns.

In one embodiment, the ultrasound-based sensing element includes one ormore rows and one or more columns, disposed in an overlapping manner,such as rectilinearly, with the effect of identifying one or more forcesensing elements at each intersection of a particular such row and aparticular such column. This has the effect that force of touch can bedetermined independently at each particular one such force sensingelement.

Similarly, in one embodiment, the touch sensing element includes one ormore rows and one or more columns, disposed in an overlapping manner,such as rectilinearly, with the effect of identifying one or more touchsensing elements at each intersection of a particular such row and theparticular such column. This has the effect that location of touch canbe determined independently at each particular one such touch sensingelement. In one embodiment, each touch sensing element includes a devicecapable of measuring a capacitance between the touch I/O element 2012(or more particularly, and element below the cover glass of the touchdevice 2012) and the user's finger, or other body part or touchingdevice. This has the effect that, when the user brings their finger nearto or touching the touch I/O element 2012, one or more capacitance senseelements detect the location of the user's finger, and produce one ormore signals indicating one or more locations at which the user iscontacting the touch I/O element 2012.

In one embodiment, the ultrasound-based sensing elements have their rowscoupled to one or more triggering and driving circuits (such as shown inthe figure as TX1 and TX2, corresponding to rows 1 and 2, respectively),each of which is coupled to a corresponding row of the ultrasound-basedsensing element. Each corresponding row of the ultrasound-based sensingelement is coupled to a sequence of one or more ultrasound-basedsensors. Each ultrasound-based sensor can, when triggered, emit anultrasonic pulse or other signal (such as shown in the figure as TX1 andTX1, again corresponding to rows 1 and 2, respectively), which istransmitted from the ultrasound-based sensor, through the elementsdescribed with respect to the FIG. 3A or the FIG. 3B, and to the surfaceof the cover glass.

The triggering and driving circuits generate one or more pulses whichare transmitted to the rows of the ultrasound-based sensing device, eachof which is coupled to a corresponding row of individualultrasound-based sensing elements. Similarly, in one embodiment, theindividual ultrasound-based sensing elements have their columns coupledto one or more sensing and receiving circuits, each of which is coupledto a corresponding column of the ultrasound-based sensing device.Collectively, this has the effect that one or more rows of theultrasound-based sensing device are driven by corresponding triggeringsignals, which are coupled to one or more columns of theultrasound-based sensing device, which are sensed by correspondingreceiving circuits.

When the ultrasonic pulse reaches the front surface of the cover glass,it would be reflected by the interface between the cover glass and theuser's fingertip, or other part of the user's body, or other touchingelement (such as a soft-ended stylus or similar device). This can havethe effect that the ultrasonic pulse would be reflected, at least inpart, back to the ultrasound-based sensor which emitted that ultrasonicpulse. The reflected ultrasonic pulse is received by one or moreultrasound-based sensors, including the ultrasound-based sensor whichemitted that ultrasonic pulse, with the effect that when the usertouches the touch I/O element 2012, a signal is received which isresponsive to the force of touch impressed on the cover glass by theuser.

One or more such reflections from the interface between the frontsurface of the cover glass and either the air or the user's finger canbe identified by the columns of the ultrasound-based sensing element(such as shown in the figure as Vout A, Vout B, and Vout C,corresponding to columns A, B, and C, respectively). Each such column iscoupled to a sense amplifier, such as shown in the figure including areference voltage Vref (such as a grounding voltage or other referencevoltage), an amplifier, and a feedback impedance element (such as acapacitor, resistor, or combination or conjunction thereof, orotherwise). Although each sense amplifier is shown in the figure ascoupled to only one sensing element, in the context of the invention,there is no particular requirement for any such limitation. For example,one or more such sense amplifiers can include a differential senseamplifier, or other sense amplifier design.

In one embodiment, each sense amplifier is disposed so that it generatesa relatively maximal response in those cases when the ultrasonicreflection from the interface between the front of the cover glass andthe user's finger is due to a force directly above the force senseelement. This has the effect that when the force sense element receivesa force of touch from the user, the relatively maximal response to thatforce of touch impressed on the cover glass by the user is primarilyfrom the ultrasound-based sensing element at the individual row/columnassociated with the location where that force of touch is relativelymaximal. To the extent that force of touch impressed on the cover glassby the user is also impressed on other locations on the cover glass, theultrasound-based sensing element at the individual row/column associatedwith those other locations would also be responsive.

In one embodiment, each sense amplifier is also disposed so that itgenerates a relatively minimal response in those cases when theultrasonic reflection from the interface between the front of the coverglass and the user's finger is due to a force from a location relativelyfar from directly above the force sense element. For example, in thecase that the ultrasonic reflection is from a portion of the ultrasonicpulse which radiates at an angle from the ultrasound-based sensor, andis similarly reflected back at that angle, the arrival time of thatultrasonic pulse would be sufficiently different from a directup-and-down reflection that the sense amplifier can be disposed todisregard that portion of the reflection of the ultrasonic pulse. Thishas the effect that the sense amplifier can be disposed to only respondto those cases when force of touch is impressed on the cover glass bythe user directly above the sense amplifier.

For example, an ultrasonic pulse can be generated by a triggering pulsefrom driving circuit, such as TX1 or TX2, with the effect of providing afirst set of (unwanted) reflections and a second set of (wanted)reflections, one set for each of Vout A, Vout B, and Vout C. Theunwanted reflections might be responsive to reflections from otherultrasonic pulses, from ultrasonic pulses that are reflected fromelements other than the interface between the front of the cover glassand either the air or the user's finger, or otherwise. For example, theunwanted reflections might occur at a time after the triggering pulsefrom driving circuit, such as less than about 450 nanoseconds after thetriggering pulse, but before an expected time for the ultrasonic pulseto travel to the front of the cover glass and be reflected, such as morethan about 450 nanoseconds after the triggering pulse. In such cases,the receiving and sensing circuits would be disposed to decline torespond to those reflections which are not within the expected window oftime duration for a response from the correct force sensing element.

In one embodiment, the touch I/O element 2012 can include a capacitivetouch sensing device, which can determine a location, or an approximatelocation, at which the user contacts, or nearly contacts, the touch I/Oelement 2012. For example, the capacitive touch sensing device caninclude a set of capacitive touch sensors, each of which is disposed todetermine if the user contacts, or nearly contacts, the touch I/Oelement 2012 at one or more capacitive touch sensing elements.

In one embodiment, the touch I/O element 2012 can combine informationfrom the capacitive touch sensing device and the ultrasound-based forcesensing device, with the effect of determining both a location of touchand a force of touch by the user.

In one embodiment, the touch I/O element 2012 can maintain theultrasound- based force sensing device in a relatively dormant state,with the effect of reducing ongoing power use, until such time as thecapacitive touch sensing device indicates that there is a contact ornear contact by the user on the touch I/O element 2012. For a firstexample, once there is a contact or near contact by the user on thetouch I/O element 2012, the touch I/O element 2012 can activate theultrasound-based force sensing device, with the effect that theultrasound-based force sensing device need not draw power at times whilethe user is not contacting the touch I/O element 2012. For a secondexample, once there is a contact or near contact by the user on thetouch I/O element 2012, the touch I/O element 2012 can activate aportion of the ultrasound-based force sensing device associated with thelocation where the contact or near contact occurs, with the effect thatonly those portions of the ultrasound-based force sensing device needdraw power only at locations which are associated with places where theuser is contacting the touch I/O element 2012.

Timing Diagram

In some embodiments various components of the computing device and/ortouch screen device may be driven or activated separately from eachother and/or on separate frequencies. Separate drive times and/orfrequencies for certain components, such as the display, touch sensor orsensors (if any), and/or force sensors may help to reduce cross-talk andnoise in various components. FIGS. 7A-7C illustrate different timingdiagram examples, each will be discussed in turn below. It should benoted that the timing diagrams discussed herein are meant asillustrative only and many other timing diagrams and driving schemes areenvisioned.

With respect to FIG. 7A, in some embodiments, the display 14 and theforce sensor 18 may be driven substantially simultaneously, with thetouch sensitive component 1001 being driven separately. In other words,the driver circuits for the force sensing device 18 may be activatedduring a time period that the display is also activated. For example,the display signal 30 and the force sensing signal 34 may both be onduring a first time period and then may both inactive as the touchsensing device signal 32 is activated.

With respect to FIG. 7B, in some embodiments, the touch and forcedevices may be driven at substantially the same time and the display maybe driven separately. For example, the display signal 40 may be set high(e.g., active) during a time that the touch signal 42 and the forcesignal 44 may both be low (e.g., inactive), and the display signal 40may be low while both the touch signal 42 and the force signal 44 arehigh. In this example, the touch signal 42 and the force signal 44 mayhave different frequencies. In particular, the touch signal 42 may havea first frequency F1 and the force signal 44 may have a second frequencyF2. By utilizing separate frequencies F1 and F2, the computing devicemay be able to sample both touch inputs and force inputs atsubstantially the same time without one interfering with the other,which in turn may allow the processor to better correlate the touchinputs and the force inputs. In other words, the processor may be ableto correlate a force input to a touch input because the sensors may besampling at substantially the same time as one another. Additionally,the separate frequencies may reduce noise and cross-talk between the twosensors. Although the example in FIG. 7B is discussed with respect tothe force and touch signals, in other embodiments each of the drivesignal, the touch signal, and/or the force signal may have separatefrequencies from each other and may be activated simultaneously orcorrespondingly with another signal.

With respect to FIG. 7C, in some embodiments, various components in thecomputing device may be driven separately from one another. For example,the display signal 50 may be driven high, while both the touch signal 52and the force signal 54 are low. Additionally, the touch signal 52 maybe high while both the force signal 54 and the display signal 50 are lowand similarly the force signal 54 may be high while both the displaysignal 50 and the touch signal 52 are low. In these examples, the forcesignal's active period may be positioned between the active periods ofthe display and the touch sensor. In other words, the force sensor 18may be driven between the display being driven and the touch sensorsbeing driven. In these examples, each of the devices may be active atseparate times from one another, thereby reducing inter-system noise. Insome embodiments, the force sensor may have a shorter drive time thanthe display or touch signals; however, in other embodiments, the forcesensor may have a drive time that is substantially the same as or longerthan the display and/or touch sensor.

Alternative Embodiments

After reading this application, those skilled in the art would recognizethat techniques for performing ultrasonic-based force sensing and touchsensing, particularly in a touch device, and using information gleanedfrom or associated with ultrasonic-based force sensing and touch sensingto perform methods associated with touch recognition, touch elements ofa GUI, and touch input or manipulation in an application program, areeach responsive to, and transformative of, real-world events, andreal-world data associated with those events, such as force sensing dataand touch sensing data received from a user's touch activity, andprovides a useful and tangible result in the service of operating atouch device. Moreover, after reading this application, those skilled inthe art would recognize that processing of force sensing data and touchsensing data by a computing device includes substantial computer controland programming, involves substantial records of force sensing data andtouch sensing data, and involves interaction with force sensing hardwareand touch sensing hardware and optionally a user interface for usingforce sensing information and touch sensing information.

Certain aspects of the embodiments described in the present disclosuremay be provided as a computer program product, or software, that mayinclude, for example, a computer-readable storage medium or anon-transitory machine-readable medium having stored thereoninstructions, which may be used to program a computer system (or otherelectronic devices) to perform a process according to the presentdisclosure. A non-transitory machine-readable medium includes anymechanism for storing information in a form (e.g., software, processingapplication) readable by a machine (e.g., a computer). Thenon-transitory machine-readable medium may take the form of, but is notlimited to, a magnetic storage medium (e.g., floppy diskette, videocassette, and so on); optical storage medium (e.g., CD-ROM);magneto-optical storage medium; read only memory (ROM); random accessmemory (RAM); erasable programmable memory (e.g., EPROM and EEPROM);flash memory; and so on.

While the present disclosure has been described with reference tovarious embodiments, it will be understood that these embodiments areillustrative and that the scope of the disclosure is not limited tothem. Many variations, modifications, additions, and improvements arepossible. More generally, embodiments in accordance with the presentdisclosure have been described in the context of particular embodiments.Functionality may be separated or combined in procedures differently invarious embodiments of the disclosure or described with differentterminology. These and other variations, modifications, additions, andimprovements may fall within the scope of the disclosure as defined inthe claims that follow.

We claim:
 1. A touch I/O device, including a touch sensor capable ofproviding touch location information, the touch location informationincluding a location at which a contact or near-contact occurs; anultrasonic force sensor capable of providing force of touch information,the force of touch information including a measure of an amount of forcepresented at the location at which the contact or near-contact occurs; adrive circuit coupled to the touch sensor and to the ultrasonic forcesensor, the drive circuit configured to: drive the touch sensor at afirst time and at a first frequency; and drive the ultrasonic forcesensor at least one of at a second different time or at a seconddifferent frequency; a sense circuit coupled to the touch sensor and tothe ultrasonic force sensor, and capable of combining information fromthe touch sensor and from the ultrasonic force sensor.
 2. The touch I/Odevice as in claim 1, wherein the ultrasonic force sensor comprisesfirst ultrasonic force sensor elements and second ultrasonic forcesensor elements, a set of intersections of the first ultrasonic forcesensor elements and the second ultrasonic force sensor elements definingone or more locations where the ultrasonic force sensor is capable ofproviding additional touch location information.
 3. The touch I/O deviceas in claim 2, wherein the first ultrasonic force sensor elements andthe second ultrasonic force sensor elements are disposed in asubstantially rectilinear pattern.
 4. The touch I/O device as in claim1, wherein: the ultrasonic force sensor comprises first ultrasonic forcesensor elements and second ultrasonic force sensor elements disposed ina pattern; a coupling of a particular one of the first ultrasonic forcesensor elements and a particular one of the second ultrasonic forcesensor elements designates one or more locations where the ultrasonicforce sensor is capable of providing the force of touch information; andthe coupling of the particular one of the first ultrasonic force sensorelements and the particular one of the second ultrasonic force sensorelements provides a circuit, the circuit providing an ultrasonic signal,the ultrasonic signal being responsive to a measure of force of touch.5. The touch I/O device as in claim 1, wherein the touch sensorcomprises first touch sensor elements and second touch sensor elements,a set of intersections of the first touch sensor elements and the secondtouch sensor elements defining one or more locations where the touchsensor is capable of providing the touch location information.
 6. Thetouch I/O device as in claim 5, wherein the first touch sensor elementsand the second touch sensor elements are disposed in a substantiallyrectilinear pattern.
 7. The touch I/O device as in claim 1, wherein: thetouch sensor comprises first touch sensor elements and second touchsensor elements, a set of intersections of the first touch sensorelements and the second touch sensor elements defining one or morelocations where the touch sensor is capable of providing the touchlocation information; the ultrasonic force sensor comprises firstultrasonic force sensor elements and second ultrasonic force sensorelements, a set of intersections of the first ultrasonic force sensorelements and the second ultrasonic force sensor elements defining one ormore locations where the ultrasonic force sensor is capable of providingadditional touch location information; and at least one of the firsttouch sensor elements and the first ultrasonic force sensor elements arecoupled to at least one of the second touch sensor elements and thesecond ultrasonic force sensor elements.
 8. The touch I/O device as inclaim 7, wherein the first touch sensor elements and the firstultrasonic force sensor elements substantially overlap.
 9. The touch I/Odevice as in claim 1, wherein: the touch sensor comprises capacitivetouch sensors disposed in a pattern, the pattern including first touchsensor elements and second touch sensor elements; a coupling of aparticular one of the first touch sensor elements and a particular oneof the second touch sensor elements designates one or more locationswhere the touch sensor is capable of providing the touch locationinformation; and the coupling of the particular one of the first touchsensor elements and the particular one of the second touch sensorelements provides a circuit, the circuit being responsive to a measureof capacitance responsive to the contact or near-contact.
 10. The touchI/O device as in claim 1, wherein a measure of reflection of a signalgenerated by the ultrasonic force sensor at a designated time indicatesthe force of touch information.
 11. A method, including operations of:driving a touch sensor at a first time and a first frequency; driving anultrasonic force sensor at a second time and a second frequency, whereinat least one of the second time does not overlap with the first time orthe second frequency is distinct from the first frequency; providingtouch location information in response to the touch sensor, the touchlocation information including a location at which a contact ornear-contact occurs; providing force of touch information in response tothe ultrasonic force sensor, the force of touch information including ameasure of an amount of force presented at the location at which thecontact or near-contact occurs; combining information from the touchsensor and from the ultrasonic force sensor.
 12. The method as in claim11, further comprising: providing a touch signal comprising the touchlocation information; and providing a force signal comprising the forceof touch information.
 13. The method as in claim 12, wherein the touchsignal and the force signal are provided at distinct times.
 14. Themethod as in claim 11, wherein the driving the ultrasonic force sensorcomprises causing the ultrasonic force sensor to emit an ultrasonicpulse.
 15. The method as in claim 14, wherein the providing the force oftouch information comprises measuring a reflection of the ultrasonicpulse.
 16. The method as in claim 11, wherein: the ultrasonic forcesensor comprises first ultrasonic force sensor elements and secondultrasonic force sensor elements; and further comprising providing forcelocation information based on a response at a set of intersections ofthe first ultrasonic force sensor elements and the second ultrasonicforce sensor elements.
 17. The method as in claim 11, wherein: theultrasonic force sensor comprises ultrasonic force sensor elements; thedriving the ultrasonic force sensor comprises causing the ultrasonicforce sensor elements to emit ultrasonic pulses; and the providing theforce of touch information comprises measuring a reflection of theultrasonic pulses.
 18. The method as in claim 11, wherein: the touchsensor comprises first touch sensor elements and second touch sensorelements; and the providing the touch location information is based on aresponse at a set of intersections of the first touch sensor elementsand the second touch sensor elements.
 19. The method as in claim 11,wherein: the ultrasonic force sensor comprises ultrasound generators andreceivers; the driving the ultrasonic force sensor comprises driving theultrasound generators; and the providing the force of touch informationcomprises measuring a signal generated by the receivers responsive tothe driving the ultrasound generators.
 20. The method as in claim 11,wherein: the touch sensor comprising capacitive touch sensors; themethod further comprises coupling a first touch sensor and a secondtouch sensor to designate the location at which the contact ornear-contact occurs based on a measure of capacitance in response to thecontact or near-contact.